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DETAILED ACTION 

Response to Arguments 

Applicant's arguments filed July 29, 2008 have been fully considered but they are not 
persuasive. 

Applicant submits the prior art does not teach or suggest determining an accumulated 
rotation angle using image data because Uchiyama calculates the angle of rotations from 
magnetic field data. 

In response to applicant's arguments against the references individually, one cannot show 
nonobviousness by attacking references individually where the rejections are based on 
combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re 
Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). 

The Wyman reference relied upon clearly discloses using only data obtained from the 
images to correct their orientation (col. 6, lines 61-64, col. 8, lines 23-37; col. 9, lines 43-49; col. 
10, line 48 - col. 11, lines 3; col. 11, lines 41-58, also see pages 4 and 5 of the previous Office 
Action) and therefore disclosed or suggested using the image data to determine a rotation angle 
between arbitrary images. However, Wyman, while teaching the correction could be extended to 
bring any arbitrary number of images into alignment (col. 9, lines 1-6), limited the disclosure to 
describing alignment of two images. Therefore, Wyman did not disclose any further angle 
notation or calculation as it would pertain to correction additional images. Uchiyama was relied 
upon by the examiner as teaching an encapsulated endoscope system that captured a plurality of 
images having different rotational information and calculating the angle of rotation between 
them to place them in the same orientation. Uchiyama expressed the angles of rotation between 
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arbitrary images as an accumulated rotation angle (page 3, paragraphs 59-63). The combination 
articulated by the examiner was not to substitute the image correction taught by Wyman with 
that of Uchiyama (which requires the magnetic field data) but the that the rotation angles 
determined by Wyman between the reference image, orientation corrected images and arbitrary 
yet to be corrected images could be expressed as an accumulated rotation angle as taught by 
Uchiyama. The extent to which Uchiyama's disclosure of accumulated rotation angle was relied 
upon was not to teach that of using the magnetic field data to determine rotation angle, but 
merely the way of expressing the relationships of the angles between the images. The 
combination of Wyman and Uchiyama as suggested by the examiner would predictably result in 
using the techniques of Wyman to align endoscope images and expressing the angles of rotation 
between the images as accumulated rotation angles as taught by Uchiyama. 

With regard to applicant's argument to the recitation of "where the second arbitrary 
image and the reference image need not have a spatially overlapping area," again, this language 
does not explicitly preclude the images from having spatially overlapping regions. The language 
is suggestive that they aren't required to have overlapping regions. The language does not insist 
that the images not have spatial overlap, nor does it define in what dimension of space, and to 
what relative space, they need not have spatial overlap. Could they overlap in one spatial 
dimension and not in another? Can they overlap in any spatial dimension, or none? Furthermore, 
the technique for determining a rotation angle between images disclosed in applicant's 
specification is that of point pair searching and matching (pages 10-13). The orientation 
correction of images not having any spatial overlap (which are never explicitly referred to as 
such) is obtained through the accumulated rotation angle. That is, intermediate adjacent images 
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have some amount of spatial overlap with the reference image and the arbitrary second image, 
their orientation correction and rotation angles are used to determine the angle between the 
reference and the arbitrary second image (page 13, Fig. 5). If the method of Wyman were 
extended to align more than two images such as the endoscope images taught by Uchiyama and 
utilized the calculation and expression of accumulated rotation angle as taught by Uchiyama, the 
combination of their teachings clearly discloses or suggests all requirements and language of the 
claims, including the "need not spatially overlapping" suggestion. 

Therefore, the combination of Wyman and Uchiyama is not deficient in its disclosure or 
suggestion of all the limitations of the presently claimed invention and the rejections under 35 
U.S.C. 103 are maintained. 

Claim Rejections - 35 USC §103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

Claims 1, 3-9 and 11-18 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Wyman in view of US 2003/0229268 to Uchiyama et al. (Uchiyama). 

Regarding claims 1 and 17, Wyman discloses a digital image processing method for 
automatic axial rotation correction of a sequence of in vivo images in a same set using image 
data (col. 6, lines 34-37; col. 7, lines 30-64; col. 1, lines 53-65), comprising the steps of: 
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a) selecting, as a reference image, a first arbitrary in vivo image from the sequence of in 
vivo images (col. 6, lines 40-45); 

b) using the image data to determine a rotation angle between a second arbitrary in vivo 
image selected from the sequence of in vivo images and the reference image of the sequence 
(col. 6, lines 57-61; col. 8, lines 23-37; col. 9, lines 43-49; col. 10, line 48 - col. 11, line 3; col. 
11, lines 41-58, Wyman teaches applying affine transformations including rotating the images in 
the image registration, the determination of an angle of a rotation angle between the reference 
image and the second image is inherent to the rotation transformation of registration method 
taught by Wyman.); 

c) correcting the orientation of the second arbitrary in vivo image, with respect to 
orientation of the reference image and corresponding to the rotation angle determined with the 
image data (col. 6, lines 61-64; col. 8, lines 23-37; col. 9, lines 43-49; col. 10, line 48 - col. 11, 
line 3; col. 11, lines 41-58). 

Wyman further discloses using orientation corrected images, obtained using the image 
data, to determine a rotation angle between other selected in vivo images and the reference image 
(col. 6, lines 57 - col. 7, line 3; col. 8, lines 23-37; col. 9, lines 43-49; col. 10, line 48 - col. 11, 
line 3; col. 11, lines 41-58; col. 9, lines 1-6, col. 15, lines 6-22, Wyman teaches applying affine 
transformations including rotating the images in the image registration thereby bringing an 
arbitrary number of images, into alignment. Wyman further teaches the transformations may be 
applied to a subset, convergence is determined and the steps are repeated, thus the transformed 
subset (orientation corrected images) is used to perform registration of the rest of the set (other 
in vivo images), the determination angle of a rotation angle between the reference image and the 
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second image is inherent to the rotation transformation of registration method taught by 
Wyman.); and correcting for the other selected in vivo images that do not match the reference 
image's orientation and where there exists a rotation angle between the other selected in vivo 
images and the reference image (col. 6, lines 61-64; col. 8, lines 23-37; col. 9, lines 43-49; col. 
10, line 48 - col. 11, line 3; col. 11, lines 41-58; col. 9, lines 1-6, Wyman teaches the process may 
be extended to bring an arbitrary number of images or image sets into alignment.). Wyman 
further discloses the disclosed technique can be used for registering any 2D or 3D images (col. 7, 
lines 61-64) 

While Wyman discloses bringing any arbitrary number of images into alignment, the 
explicit details of the alignment are limited to a discussion of only two images. Therefore, 
Wyman is silent with regard to an accumulated rotation angle which is used to correct other 
selected in vivo images and the reference image. Uchiyama discloses using angles of orientation 
corrected images to determine an accumulated rotation angle between other selected in vivo 
images in a sequence and a reference image from a sequence of in vivo images captured by an 
encapsulated endoscope and correcting for the other selected in vivo images that do not match 
the reference image's orientation using the accumulated rotation angle where there exists a 
rotation angle between the other selected in vivo images and the reference image, where an 
arbitrary second image need not have spatially overlapping area with the reference image (Figs. 
5 and 6; pages 3-4, paragraphs 58-73). 

Wyman and Uchiyama are combinable because they are both concerned with medical 
image processing, specifically rotating the medical images to properly align them for display. It 
would have been obvious to one of ordinary skill in the art at the time the invention was made to 
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combine the techniques taught by Wyman and Uchiyama such that the image registration 
techniques taught by Wyman could apply to images captured by an encapsulated endoscope 
system such as the one taught by Uchiyama (and the present invention) and that the method of 
Wyman could be modified to include using the orientation corrected images to determine an 
accumulated rotation angle between other selected in vivo images the sequence and a reference 
image from a sequence of in vivo images captured by an encapsulated endoscope and correcting 
for the other selected in vivo images that do not match the reference image's orientation using 
the accumulated rotation angle where there exists a rotation angle between the other selected in 
vivo images and the reference image, as taught by Uchiyama, in order to bring any arbitrary 
number images into alignment with the reference image. 

Regarding the limitation "where the second arbitrary image and the reference image need 
not have a spatially overlapping area" as recited in claim 17, the language of the claim does not 
explicitly preclude the images from a having spatially overlapping area, just that they don't 
necessarily have to. Thus the images disclosed by Wyman meet the requirements of the claim. 
Further Uchiyama' s images are a sequence of image captured by the encapsulated endoscope and 
are aligned based on the accumulated rotation angle and therefore, the first image would not need 
not to be spatially overlapping with an arbitrary other image in the sequence. 

As to claim 18, while having different language from that of the method recited in claim 
17 and not claiming the selection of a reference image, limitations analogous to those of claim 18 
have been addressed in the in the rejection of claims 1 and 17 above. Therefore, arguments 
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analogous to those presented above for claims 1 and 17 are applicable to claim 18, the "reference 
image" of claims 1 and 17 corresponding to the "first image" recited in claim 18. 

Regarding claim 3, the combination of Wyman and Uchiyama further discloses the step 
of correcting the orientation of the arbitrary in vivo image, with respect to orientation of the 
reference image and corresponding to the rotation angle uses an accumulated correction angle 
derived from the accumulated rotation angle (Wyman, Fig. 5; col. 8, line 60 - col. 9, line 6; col. 
10, line 48 - col. 11, line 3; Uchiyama, Figs. 5 and 6; pages 3-4, paragraphs 58-73). 

Regarding claim 4, the combination of Wyman and Uchiyama further discloses the 
rotation angle measured with respect to an optical axis of an in vivo camera used to capture the 
plurality of in vivo images, and wherein the optical axis is perpendicular to an image plane and is 
parallel to the in vivo camera's travel trajectory derivative (Uchiyama, Figs, 3, 9, 10, 12, 13, 14, 
15, 16; paragraphs 6, 42, 55-58). 

Regarding claims 5 and 6, the combination of Wyman and Uchiyama further discloses 
the rotation angle is defined in a right-hand system or a left-hand system and angle is rotated 
counter-clock wise or clockwise relative to the reference image's orientation, such that the 
rotation angle is a signed value (Wyman, Fig. 8; Fig. 9E; col. 8, lines 1- 37; col. 11, line 41 - col. 
12, line 4; col. 14, lines 60-62). 
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Regarding claim 7, the combination of Wyman and Uchiyama further discloses the 
plurality of in vivo images have a plurality of feature points, and wherein the plurality of feature 
points are used for finding an orientation difference between two in vivo images (Wyman, col. 6, 
lines 46-64). 

Regarding claim 8, Wyman discloses an origin of a two-dimensional coordinate system 
of the in vivo images, thus defining an image plane, is at an image's center (Fig. 5; col. 10, lines 
48 - col. 11, line 3, Wyman clearly shows two-dimensional images oriented in an X-Y 
coordinate system, with Z coordinate identifying each image's two-dimensional plane location in 
the stack of images, with an origin at the intersection of the intersection of axes. It is clear if the 
image sets are limited to a single image (as suggested by Wyman at column 8, lines 63-65), the 
coordinate system would be limited to a two-dimensional system as there is no "stack" of 
images"), and further comprising the steps of: 

a) collecting the plurality of feature points that reside on a first image plane (Fig. 5; col. 
6, lines 46-54; col. 10, line 48 - col. 11, line 3, Wyman selects points at the corners of the 
images); 

b) finding a corresponding plurality of feature points in a second image plane (col. 6, 
lines 46-54; col. 10, lines; col. 10, line 48 - col. 11, line 3); 

c) determining whether a feature point that resides on the first image plane moves in the 
second image plane (col. 6, lines 54-57); and 

d) measuring the feature point's movement in the second image plane to determine the 
rotation angle and its direction (col. 6, lines 57-64; col. 10, line 48 - col. 11, line 3). 
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The combination of Wyman and Uchiyama does not disclose expressly collecting the 
plurality of feature points that reside on the axis of a first image plane; determining whether the 
feature point that resides on the axis of the first image plane moves off axis in the second image 
plane; and measuring the feature points movement off axis in the second image plane to 
determine the rotation angle and its direction. 

At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to modify the feature points taught by Wyman to include points that reside on the axis of 
the image plane. Applicant has not disclosed that collecting feature a plurality of feature points 
that reside on an axis of an image plane provides an advantage, is used for a particular purpose or 
solves a stated problem. One of ordinary skill in the art, furthermore, would have expected 
Applicant's invention to perform equally well with points not on the axis because applicant's 
specification recites "people skilled in the art can easily extend this analysis to points that are not 
on the axis" (page 13, lines 3-4). 

Therefore, it would have been obvious to one of ordinary skill in this art to modify the 
combination of Wyman and Uchiyama to include collecting a plurality of feature points that 
reside on an axis of a first image plane to obtain the invention as specified in claim 8. 

Regarding claims 9, 11-16, the combination of Wyman and Uchiyama further discloses a 
computer storage medium having instructions stored therein for causing a computer to perform a 
method (Wyman, col. 11, lines 18-39). 
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Conclusion 

Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to ANTHONY MACKO WEY whose telephone number is 
(571)272-7425. The examiner can normally be reached on M-F 9:00-6:00. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Matthew Bella can be reached on (571) 272-7778. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Matthew C Bella/ 

Supervisory Patent Examiner, Art Unit 
2624 
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